Tracking an object in a video includes detecting the object in the video and then tracking the trajectory that the object makes throughout the entire video. The detection and trajectory can be used to show where the object was moving and what the object was doing. However, multitarget tracking of pedestrians and sports players in a video is a difficult problem due to the presence of many similar-looking objects interacting in close proximity with each other. The resulting clutter and occlusions lead to frequent missed/false detections and incorrect associations of the pedestrians and sports players.